Magnification Laws of Winner-Relaxing and Winner-Enhancing Kohonen Feature Maps

Self-Organizing Maps are models for unsupervised representation formation of cortical receptor fields by stimuli-driven self-organization in laterally coupled winner-take-all feedforward structures. This paper discusses modifications of the original Kohonen model that were motivated by a potential function, in their ability to set up a neural mapping of maximal mutual information. Enhancing the winner update, instead of relaxing it, results in an algorithm that generates an infomax map corresponding to magnification exponent of one. Despite there may be more than one algorithm showing the same magnification exponent, the magnification law is an experimentally accessible quantity and therefore suitable for quantitative description of neural optimization principles. Self-Organizing Maps are one of most successful paradigms in mathematical modelling of special aspects of brain function, despite that a quantitative understanding of the neurobiological learning dynamics and its implications on the mathematical process of structure formation are still lacking. A biological discrimination between models may be difficult, and it is not completely clear [1] what optimization goals are dominant in the biological development for e.g. skin, auditory, olfactory or retina receptor fields. All of them roughly show a self-organizing ordering as can most simply be described by the SelfOrganizing Feature Map [2] defined as follows: Every stimulus in input space (receptor field) is assigned to a so-called winner (or center of excitation) s where the distance |v−ws| to the stimulus is minimal. According to Kohonen, all weight vectors are updated by δwr = ηgrs · (v −wr). (1) This can be interpreted as a Hebbian learning rule; η is the learning rate, and grs determines the (pre-defined) topology of the neural layer. While Kohonen chose g to be 1 for a fixed neighbourhood, and 0 elsewhere, a Gaussian kernel (with a width decreasing in time) is more common. The Self-Organizing Map concept can be used with regular lattices of any dimension (although 1, 2 or 3 dimensions are preferred for easy visualization), with an irregular lattice, none (Vector Quantization) [3], or a neural gas [4] where the coefficients g are determined by rank of distance in input space. In all cases, learning can be implemented by serial (stochastic), batch, or parallel updates. ⋆ Published in: V. Capasso (Ed.): Mathematical Modeling & Computing in Biology and Medicine, p. 17–22, Miriam Series, Progetto Leonardo, Bologna (2003).